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75. (New) The method of claim 73, wherein transmitting sensor data 
further comprises transmitting environmental sensor data. 

REMARKS ~ 

The specification and claims have been amended to place them 
in better form. Claims 1, 7, 20, 21, and 2 3-33 have been amended. 
New claims 34-75 have been added. It is respectfully submitted 
that the changes to the claims and new claims be carefully reviewed 
by the Examiner and entered prior to examination. 

The subject application generally discloses a device and 
method which overcomes shortcomings associated with self contained 
systems which do not allow two-way communications over local and 
wide area networks to other mult i -media users and devices. See, 
e.g., p. 2, lines 2-6. Specifically, a wearable portable access 
unit (PAU) 100 communicates over a wireless link to a centrally- 
located network access unit, i.e., a general purpose node 150. 
See, e.g., Fig. 1; p. 5, lines 8-9. The system provides two way 
radio, audio, and data communications between a PAU 100 and the 
general purpose or routing node 150 which is connected to one or 
more media devices through a network. See, e.g., Fig. 1; p. 2, 
lines 27-30; p. 5, lines 8-13. 

User commands are routed to these devices over a network 
through the general purpose node 150 which acts as a router for 
directing video, display, audio, and control data between PAUs 100 
and other devices coupled to the network. See, e.g., p. 5, lines 
13-16. With this configuration, processing functions can be 
removed from a "self contained" unit and completed with more 
efficient processing, media, or network devices coupled to the 
network. 
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The outgoing user commands and incoming data are formatted 
using an audio/video coder/decoder (codec) pair 200. One codec is 
part of a PAU 100, and the other codec 200 is part of another PAU 
100. Thus, the codecs serve as the interfaces to external elements 
to code outgoing signals and decode incoming signals. See, e.g., 
p. 6, lines 25-30; p. 10, lines 23-30 ; Fig. 2. While the codec is 
used for data processing and to code/decode outgoing/incoming data, 
user commands are executed by media devices, thereby resulting in 
faster, more efficient processing using wireless network 
interfaces . 

Data and user commands are routed between a user / PAU and a 
media device or other PAU through the routing or general purpose 
node. See, e.g., p. 5, lines 13-16 (general purpose node or 
router) ; Fig. 1 (illustrating communications between PAUs through 
general purpose node) ; p. 10, lines 19-30 (communications being 
routed through general purpose node) . As a result, the PAU serves 
as a data interface and format outgoing and incoming data. The 
general purpose node routes commands to media or network devices 
which execute the commands to generate a result, separate from the 
codec and PAU. See, e.g., p. 5, lines 13-15 (routing function 
performed by general purpose node); p. 6, lines 19-30 (codec 
controlling digital data stream, feeding stream to communications 
device; codec serves as interface); p. 9, lines 10-13, 25-30 (codec 
having inputs/outputs for formatting audio and video data) . The 
result or data can then be routed through the general purpose node 
to the corresponding PAU. 

The amendments to the specification merely clarify these and 
other aspects of Applicant's disclosure and are primarily directed 
to the PAU serving as an interface, the general purpose or routing 
node, and coder/decoder ("codec"). These specification amendments 
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do not add new matter, and entry of the amendments is therefore 
respectfully requested. 

Specifically, the PAU serves as a data interface since user 
commands are executed by devices which are coupled to a network. 
In other words, user commands are not executed by a PAU. Rather, 
it serves as an interface. 

Data retrieved from the network and commands issued by a user 
are routed through the general purpose or routing node. Thus, even 
to communicate with other PAUs, data is transmitted through the 
routing or general purpose node. See, e.g., Fig. 1 (communications 
between PAUs is through general purpose or routing node 150) . 

The codec in the PAUs formats the data that is sent and 
received. As explained in the specification, the codec can be a 
processor chip such as a multimedia processor chip. See, e.g., p. 
9, lines 27-30. However, the user commands are processed or 
executed by network or media devices, not the PAUs (compare media 
and network devices coupled to network in Figure 1) . 

Considering the forgoing remarks, the amendments to the claims 
and the new claims are supported by the specification as originally 
filed. The claim amendments and new claims do not add new matter, 
and entry of the claim amendments and new claims is therefore 
respectfully requested . 

Claim 1 was amended to refer to a wireless communication 
device. Claim 7 was amended to specify that the media device is 
configured to execute the command separate from the PAU. Claims 
2 0-21 and 2 3 were amended to correct a typographical error. Claims 
24-33 were amended to refer to "further" comprising. New claims 
34-75 recite additional novel and unobvious features of the subject 
application. Specifically, new claims 34-38 relate to the manner 
in which the PAU is configured and how the general purpose node 
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routes data between the PAU and the network or media device. New 
claims 3 9-60 are directed to a wireless communications interface 
system, and new claims 61-75 are directed to a method for 
communicating data over a wireless network and displaying the data 
on a data interface unit. 

Due to the number of amendments, a substitute specification 
pursuant to 37 CFR § 1.125 and MPEP § 608.1 (q) is submitted 
herewith to facilitate the prosecution of this application. The 
substitute specification is accompanied by a marked up copy showing 
the changes between the original application, as filed, and the 
substitute specification. The substitute specification does not 
contain any new matter and includes the same changes are indicated 
in the marked up copy. Applicant respectfully requests that the 
substitute specification be entered in this case. 

Attached hereto is a marked-up version of the changes made to 
the claims by the current amendment. The attached page is 
captioned " Version with markings to show changes made ." No new 
matter has been added. 

In view of the forgoing amendments and remarks, consideration 
and allowance of this application is respectfully requested, 



Respectfully submitted, 



CHRISTIE, PARKER & HALE, LLP 




Gary D . Lueck 
Reg. No. 50,791 
626/795-9900 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 
In the Claims 

1. (Amended) A communication system for communicating through a 

network , comprising : 

a general purpose node electrically connected to the network 

for providing access through the network, the general purpose node 

having a wireless communication device; 

at least one media device connected to the network; and 

a portable access unit capable of wirelessly communicating 

with the general purpose node for communicating with the media 

device through the network. 

7. (Amended) The system of claim 6, wherein the portable access 
unit is for providing commands for controlling the processor^ 
wherein the media device is configured to execute the commands 
separate from the portable access unit . 

20. (Amended) The system of claim 19, wherein each portable access 
unit is ado p t e d adapted for listing on a display the plurality of 
portable access units that are associated with the plurality of 
general purpose nodes. 

21. (Amended) The system of claim 19, wherein each portable access 
unit is adopted adapted for listing on a display the plurality of 
media devices. 

23. (Amended) A method for communicating through a network with at 
least one media device connected to the network, comprising:. 
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providing access to the network with a general purpose node 
electrically connected to the network, the general purpose node 
having a wireless communication device; and 

communicating wirelessly with the remote media device through 
the general purpose node and the network with a portable access 
unit that is in wireless communication with the general purpose 
node . 

24. (Amended) The method of claim 23, further comprising receiving 
video signals from the media device for providing video signals for 
display on the portable access unit. 

25. (Amended) The method of claim 23, further comprising 
transmitting video signals from the portable access unit to the 
media device for presenting the video signals on the media device. 

26. (Amended) The method of claim 23, further comprising 
transmitting audio signals from the portable access unit to the 
media device for presenting the audio signals on the media device. 

27. (Amended) The method of claim 23, further comprising receiving 
audio signals captured by the remote media device from the remote 
media device for presenting on the portable access unit. 

28. (Amended) The method of claim 23, further comprising 
transmitting commands from the portable access unit to the media 
device for controlling the media device. 

29. (Amended) The method of claim 23, further comprising receiving 
data captured by a sensor on the media device. 
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30. (Amended) The method of claim 23, further comprising 
dynamically associating and de-associating the portable access unit 
with the general purpose nodes . 

31. (Amended) The method of claim 30, further comprising 
presenting a list of a plurality of portable access units on a 
display that are associated with the general purpose node. 

32. (Amended) The method of claim 31, further comprising listing 
on the display a plurality of media devices associated with the 
general purpose node. 

33. (Amended) The method of claim 23, further comprising 
presenting biological data for a user of one of a plurality of 
portable access units after selecting the user's name from a list 
of users of the plurality of portable access units. 



GMV PAS433340. 2-*-5/10/02 3:40 PM 



19 



WIRELESS AUGMENTED REALITY COMMUNICATION SYSTEM 
RELATED APPLICATIONS 
This application is based on provisional patent application serial number 60/1 15,993 filed 
January 15, 1999. 

GOVERNMENT LICENSE RIGHTS 

The U.S. Government has certain rights in this invention pursuant to NAS7- 1 407 awarded by NASA. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The invention is a wireless augmented reality system (WARS) that leverages communications 
and multimedia information processing microelectronics, along with displays, imaging sensors, 
biosensors, and voice recognition to provide hands-free, tetherless, real-time access and display of 
network resources, including video, audio and data. 

2. Description of the Prior Art and Related Information 

Online instruction manuals are becoming more prevalent in the industrial and everyday 
environment. These electronic technical manuals (ETM) may be interactive. Just as with printed 
manuals, ETMs may become very difficult to use and maintain in these environments where 
elements of an environment, such as dust, chemical or general harshness may be detrimental to the 
electronics and storage devices used to display and operate the ETM. Further, it is not always 
possible for a worker who requires access to an ETM to stop work to consult ETM. 

These problems are multiplied in extraterrestrial environments such as a space shuttle or a 
space station. During intra and extra vehicular activities, it may be virtually impossible to access a 
traditional keyboard and computer display to access an ETM. For example, during a satellite repair 
mission, it would not be practical for an astronaut in a bulky extravehicular space suit to type 
commands on a keyboard to view a display in the extreme environment of outer space where the sun 
glare may make viewing impossible. 

Hands-free portable computers have been implemented in an attempt to solve some of these 
problems. For example, U.S. Patent Nos. 5,305,244 and 5,844,824 describe systems in which a 
head-up display and voice recognition is implemented in a portable computer for displaying ETM. 
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However, these systems, being a single user-to-computer paradigm, do not allow multiple-user 
access to multiple computers, multimedia devices or nodes on a network for accessing arbitrarily- 
selected data channels. Further, these previously-described systems are self contained and their data 
storage needs to be updated periodically to be sure that the latest data is displayed. Further, these 
systems do not allow two-way communication over local and wide area networks to other mutli- 
media users and devices, and do not provide real-time biomedical information about the physical 
condition of the user. 

There is thus a need for a wireless, wearable communications system allowing two-way 
voice, video and data communication between local users and to remote users and devices over 
network nodes, along with tetherless real-time monitoring of the local user's physical condition. 
SUMMARY OF THE INVENTION 

The system solves the above problems with prior art systems with an adaptive wireless 
remote access network comprised of small individual portable access units which communicate with 
the general purpose node through a wireless cellular connection. Media devices are coupled to the 
network and execute user commands. Interlinked general purpose nodes support communications 
across different habitat modules or internal-to-extravehicular communications, in the case of the 
space environment; terrestrial wired networks such as the Internet can serve as the interconnection 
of remotely scattered access nodes in an industrial, commercial or home environment application. 

The system may provide shuttle and international space station astronauts with tetherless, on- 
demand access to data channels from multimedia devices such as cameras or audio sensors 
associated with other persons or in a stand-alone configuration, and multimedia or data display from 
a networked computer terminal and to the equipment control capabilities which may be available 
through that computer. Transparent to such access, the system can maintain a data channel for 
monitoring an astronaut's health or environment via in-situ sensors. Though this system may be used 
for the shuttle and the international space station, the system has uses in many possible applications 
related to medical, industrial, and commercial areas. 

The invention is a personal communications system designed especially for the space shuttle 
or station environment to provide cellular communications access throughout the vessel with video, 



2 




39257/GDL/C766 Substitute Specification 

audio, data and computer connect services. A small, wearable portable access unit (PAU) serves as 
a data interface and communicates over a wireless high-rate link to a centrally-located network 
access unit, called a general purpose node herein, for routing data and commands to/from the user. 
The system backbone provides 2-way video, 2-way audio, and a multi-purpose data channel between 
the PAU and general purpose node. One embodiment of the PAU used for personal communication 
has an attached headset with video display, audio feed and camera, which together may be used for 
audio or video teleconferencing. When used as a virtual terminal to a computer in the network, the 
user is able to view and manipulate imagery, text or video, using voice commands to control terminal 
operations. This can be accomplished by submitting a command through the general purpose node 
to a device which executes the command. 

Using the system, an astronaut may efficiently operate and monitor computer-controllable 
activities inside or outside the vehicle or station. Hands-free access to computer-based instruction 
texts, diagrams and checklists replaces juggling manuals and clipboards, and tetherless computer 
system access allows free motion throughout a cabin while monitoring and operating equipment. 
Along with voice commands, an integrated "touchpad" on the PAU may be used to submit user 
commands through the general purpose node and to any media devices, for remote computer control 
through a sensor data channel. This return data channel may also be used to transmit other control 
data from a three-D mouse or data glove input device, through the general purpose routing node, to 
control a device coupled to the routing node through the network, allowing the real-time video 
display to remotely monitor and control robotic cameras or manipulators. 

Concurrent with information provided to the astronaut, the system also allows external 
observation of the astronaut's situation; personal biological or other sensors can send back 
continuous telemetry through personal access unit and general purpose node. A miniature camera 
integrated into the headset provides real-time video of the wearer's field of view to remote observers. 
In this way, for example, a principal investigator located on Earth may consult with a payload 
specialist on the operation or troubleshooting of their equipment. 

The system provides wireless high-rate data exchange. The radio link is adapted to operate 
within a high-interference, high-multipath environment of a space shuttle or space station module. 
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Radio frequency (RF) links do not require visual line-of-sight to operate, but the metal walls and lack 
of RF absorbers, combined with moving human bodies, creates an enormous potential for destructive 
self-interference of the radio signals. The integrated radio and multimedia data processing 
technology provides for efficient and high-quality video and audio data compression for noisy indoor 
communications channels. The system supports multiple-user access for video, audio, and sensor 
data services in the wireless coverage area. Potential applications of the system are in any 
environment where heads-up, hands-free information retrieval or multimedia communications access 
improves efficiency including tetherless operations/monitor consoles, remote consultations in 
medical or maintenance procedures, and hazardous/confined space activities. There are also in-the- 
home entertainment/communications applications. 

Similar to the space extravehicular activities applications, bio-isolation suits have similar 
operation constraints to space suits. They are worn commonly where there are chemical or biological 
contaminates, and any extraneous materials brought into a chamber, such as clipboards or 
documents, also present a contamination risk. A unit suitably modified for use in a space suit could 
be used in this situation. This allows the user to use a computer (log data, use a check list, etc.), to 
communicate with colleagues, including providing first-hand video of work in progress, and to 
maintain constant monitoring of the health of the user. 

An extension of the medical applications areas would be in remote telemedicine. Many 
medical diagnostic and treatment tools are being made portable and rugged enough to be taken to 
remote sites. Some examples are an ultrasound unit that is the size of a backpack, an entire intensive 
care unit of equipment built into a stretcher, and a trauma pod built into a cruise missile. For many 
of these devices, CRT or LCD panels comprise a significant amount of the bulk and weight of the 
devices. The system of the present invention may provide a replacement for the CRT or LCD panel 
as well as an interface to the control system of the device, while providing communications access 
through an interface to the remote site's existing communications equipment. 

Industrial applications include use by inspection or maintenance crews in remote or 
dangerous environments such as oil refineries, drilling figs, power plants, etc., where the personnel 
can move around with their hands and peripheral vision free to attend to their own safety and tasks. 
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They would be in constant contact with the information they needed and any technical assist could 
be given by individuals looking at the return video images from the user. 

An example of a commercial application is for mission control and other operations 
personnel who presently must sit at a display console for hours at a time. These individuals could 
make use of the system of the present invention to increase their mobility and efficiency. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a diagrammatic illustration of the components of the system of the present invention; 
Fig. 2 is block diagram illustrating communications components used by the personal access unit 
and general purpose node of the system of Fig. 1; and 

Fig. 3 is a flowchart illustrating a method performed using the system of Fig. 1 . 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

With reference to Fig. 1, a diagram illustrating components of the system of the present 
invention is shown. The system may comprise small pager-like devices called portable access units 
100. The portable access units 100 serve as different "multimedia" data interfaces for display, 
camera, audio and sensor operation. Another embodiment of the portable access unit 1 00a comprises 
a wearable headset and microphone assembly 102a. 

The portable access units 100- 100a interface directly through wireless link with a network 
through a general purpose node 150. The general purpose node 150 allows wireless-to-wire 
communication with a local network 1 70. The local area network 1 70 may be electrically connected 
to a wide area network or Internet 172 in order to connect to remote local area networks 174. 
Alternatively, the general purpose node 1 50 may be directly connected to the wide area network 1 72. 
The general purpose node 150 may thus act as a router for routing video, display, audio and control 
data packets between the portable access units 100 and other, or remote, portable access units 100 
or remote media devices 125, 180, etc. connected to the networks 170-174. The connection with a 
network 170-174 may occur directly in electrical connection with one of the networks 170-174, or 
in wireless communication through a remote general purpose node 1 5 0a that is electrically connected 
to the network. The portable access units 100 may provide communication to and from remote media 
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devices such as computers 180-182 through the general purpose node which routes the 
communication to the corresponding media device. The computers 180-182 may be running 
specialized client software or certain commercial multimedia Internet software products such as 
video conferencing products that adhere to the industry standard H.323 for multimedia data transfer. 

Each portable access unit 100- 100a may dynamically associate with the closest general 
purpose node 150- 150a when it is logged on to the network 170-174 or is connected thereto. Each 
general purpose node 1 50- 1 50a records the associations and registers each portable access unit 1 00- 
100a on a list of connections associated with the particular general purpose node 1 50-1 50a. The list 
of connections is stored in a random access memory device included in the general purpose node 
1 5 0- 1 50a. When a portable access unit 1 00 is logged off or disconnected from the network 1 70- 1 74, 
it is disassociated from the particular general purpose node 1 50- 1 50a that it was associated with, and 
thus, is removed from the list of connections. 

As shown on an example selection list screen 190 that may be presented on a display 102 or 
headset 1 02a on any of the portable access units 1 00- 1 00a, the user can set up a video, audio, or data 
link with any other portable access unit 100- 100a or remote media device 125, 180, etc, that is 
logged onto a network 1 70-1 74 through the general purpose node. The headset 102a may comprise 
a heads-up display ( 1 20 in Fig. 2) inside a headset embodying a transparent color LCD device. Using 
control keys or voice commands, a user of the portable access unit 100- 100a may select a local or 
remote portable access unit 1 00- 1 00a on a selection list 1 90 of other portable access units 1 00- 1 00a 
or media devices 125,180. The selection list 1 90 comprises a combination of the lists of connections 
stored on all of the general purpose nodes 1 50-1 50a. Users may further access a nameserver located 
on the access node 150 for locating remote unfamiliar portable access units 100- 100a or remote 
media devices. 

By selecting entries from the selection list 190, users may communicate with portable access 
units 100- 100a or various media devices such as cameras 125, internet phones 104, one or more 
computers 180-182 located throughout the networks 170-174, through the general purpose node. 
The general purpose node routes the communication from the portable access unit, to a network or 
media device, or to another portable access unit. A user may further select from the list 190 user 
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names representing users of other portable access units 100 that are logged in and associated with 
remote general purpose nodes 150a connected to the networks 170-174. 

With reference to Fig. 2, the components of the access node 150 and the wearable headset 
embodiment of the portable access unit 1 00a is shown. The portable access unit includes a display, 
an encoder and a decoder or "codec", and a two-way communications device or transceiver. The 
general purpose access node also includes a two-way communications device 202, such as a 
transceiver. As a result, the portable access unit and general purpose node can communicate through 
a wireless connection through respective transceivers. Data processing functions are implemented 
in the form of an audio/video coder/decoder (codec) pair 200, one codec 200 comprising part of the 
portable access unit 100a, and the other codec 200 being part of another portable access node 100a 
or remote media device for which it is desired to exchange signals. At a portable access node, the 
codec 200 controls a digital data stream by formatting a command by the user into a form that can 
be transmitted by the communications device 202, which is implemented as an RF modem 
transceiver pair with an equivalent communications device 202 located in the general purpose access 
node. The general purpose node is configured to receive the formatted command from the 
transceiver of the portable access unit and transmit and route it using its own transceiver over the 
wireless connection. The codecs 200 serve as the interfaces to the external elements (including 
possibly the user display 102a and the sensor 104) on both sides of the communication continuum 
comprising the communications device 202 of the general purpose node 150, an internal network 
interface protocol circuit 152, the external networks 1 70- 1 74 and the electrical connection or general 
purpose access node connection to the desired remote portable access node or media device. The 
internal network interface protocol circuit 152 may comprise an Ethernet chip, memory and a 
network protocol chip. With this architecture, the system addresses the issues of multiple- access and 
data channel quality, through the implementation of the communications device 202. Multiple 
implementations of the communication device 202 in the general purpose node 150 allows for 
multiple simultaneous communication links with a plurality of portable access units 100- 100a 
through the general purpose node 150. 
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With the base functionality of the communications device 202 and codec subsystem 200, the 
architecture provides flexibility in utilization of different external components such as different 
headset 102a configurations, sensor 104 packages, and network interface 152 capabilities. 

The communication device 202 is designed to operate in a high multipath space station or 
terrestrial indoor environment while being able to support multiple users at high, multimedia-type 
bandwidths. Thus the communications device's 202 physical (PHY) and media access (MAC) layers 
in combination support multiple access, dynamic network association, channel error rates of 
broadcast video quality (lxlOe-6), and data rates up to broadcast-quality video bandwidths (on the 
order of 768 kbps per user (one-way)). Modulation to achieve this performance will be differential 
phase-shift keying, of binary or higher order (quadrature or high-order quadrature amplitude 
modulation); the order chosen reflects the necessary user data volume to be supported within fixed, 
FCC-specified bandwidth allocations. Orthogonal frequency division multiplexing, code division 
multiple access, and frequency hopping/time division multiple access may be used for achieving 
multiple access. Spread spectrum, channel equalization, antenna diversity and retransmission 
techniques may also be used for improving the reliability of the communications link. Through a 
combination of these technologies, two-way multimedia channel throughputs can be achieved for 
each of multiple simultaneous users. A variety of RF frequencies may be used, but the determining 
factor in frequency band selection is the availability in the band of a relatively large amount of 
spectrum in the space station or FCC terrestrial allocations, allowing the transmission of compressed 
video. Ranges in the 2.5 to 5.7 band range are preferable due to the FCC bandwidth available, the 
compactness of RF elements required at these frequencies, and the potentially low amount of 
interference that will be sustained. The RF front end of both the portable access unit 100-1 00a and 
general purpose node 15 0-1 50a may be interchangeable with different frequency front ends for 
system use in different 2 frequency bands. 

Low-rate, single user implementations of the communications system may be effected 4 
through adapted commercial wireless-LAN type products following the FCC 802.1 1 standard 5 such 
as a frequency-hopping 2.4 GHz wireless LAN transceiver by Waveaccess, Inc of 6 Wellesley, MA, 
or direct-sequence spread-spectrum 2.4 GHz wireless LAN chipset by Harris 7 Prism of Melbourne, 
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FL. These radio implementations, as with commercial implementations of 8 the industry-proposed 
Bluetooth and HomeRF standards, will be limited in user access and 9 overall throughput, however, 
and therefore unsuitable to real-time video teleconferencing for 10 multiple users. The preferred 
embodiment for full capability is to implement the communications devices* physical and media 
access control layers in custom ASIC circuits 

allowing for support of all system capabilities within microelectronics architecture for small size 1 3 
and low power draw, providing pager-type form factor of wearable personal access units 1 00 1 4 1 00a. 

The communications device 202 comprises a buffer memory and a radio frequency front 1 6 
end. Data modulation/demodulation circuits and error detection/correction protocol circuits are 17 
further included. Various combinations of these circuits may be obtained from Proxim of 18 
Sunnyvale, California, Harris of Melbourne, Florida and Stanford Telecom of Stanford, 19 
California. Alternatively, all of the various circuitry may be implemented with an application 20 
specific integrated circuit (ASIC), or a combination of an ASIC and discrete elements for size 2 1 and 
weight efficiency. 

Three classes of headsets 102a maybe used: hi-resolution military systems which are CRT 
based and may be provided by Honeywell of Morristown, New Jersey, or Hughes Network Systems 
of San Diego, California; medium resolution industrial systems which are CRT or LED based 
scanners and may be provided by Intervision of Santa Clara, California; orlowto medium resolution 
entertainment systems which are color viewfinder LCD based systems that may be 27 supplied by 
Virtual Vision of Redmond, WA (the V-C AP and E-GLASS), Sony Europe of 28 Hampshire, United 
Kingdom (GLASSTRON VISOR) or Olympus of San Jose, California. 29 Typical headset display 
120 specifications for the portable access unit 1 00a include the 30 following: 

-RESOLUTION: Comparable at least to VGA (640x480) or better to 1280x 1024 w/off-the- 
shelf display & I/O configuration 

- DISPLAY: >10 FL/day, Display Bright.Ratio:>2,Bnghtness range:200Mmax 

- FOV: 40-60 deg ,Gray scale: >12 

- EyeRelief: 20-26 mm TSP, 14/10mm(on/off-axis) exit pupil 

- Unif: 2:1 across 2/3 FOV, GLARE:< 2.5% image content,PixelContrast:25 
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- FOCUS: Hands off, Obs:, % look- around ,Diopter range: +2, 

- Mag: 1+ p5%,Cont: >95%, motion sensor 10°cone, Inter. Eye. adj: 52-72mm 

- Image Enhan & IFF: Weaponsight, motion sensor and computer interface 

The audio/video codec 200 in a portable access unit 1 00- 100a or other client device is based 
around a single chip, standards-based codec that facilitates transmission of user commands over the 
network and receiving data or results in response thereto. 

When transmitting data, the codec formats data or commands for transmission by the 
communications device over the wireless connection to the general purpose node. More specifically, 
the codec accepts analog or digital audio and video (i.e. NTSC or VGA), compresses this input, and 
multiplexes the compressed data with an external data stream. The preferred industry standards are: 
ITU H.263 based video, ITU G.722 based audio, and ITU H.221 based multiplexing. The audio 
video codec 200 in the portable access unit 100- 100a can establish a link with a similar audio/video 
codec 200 associated with another portable access unit 100-100a oraremote media device 104, 125, 
180 or 182_through the general purpose node. 

When receiving data over the wireless connection through the general purpose node, the 
codec 200 in the portable access unit 1 00a formats the data routed through the general purpose node 
by decompressing or demultiplexing the data and outputting the received data from the device for 
which the link was established to the user on the portable access unit. 

The interface between the codec 200 and communication device 202 as well as between the 
communication devices 202 of the general purpose node 150-1 50a and portable access unit 1 00- 1 00a 
operate two-way with a high bandwidth suitable for transmitting video. Of this bandwidth, the audio 
portion utilizes up to 64 kbps and the data from the sensor 104 utilizes the required amount for the 
type of sensor 104, with the remainder allocated to compressed video. The quality of the video at 
these data rates in excess of 128 kbps is at least equivalent to video teleconferencing quality video. 

The audio/video codec 200 portion of the portable access unit 1 00- 1 00a may further comprise 
video input and output ports, audio input and output ports, data input and output ports. The above- 
mentioned multimedia processor chip of the codec can format data retrieved through the general 
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purpose node. The codec can be used for packaging signals for data compression and decompression 
for transmission. Exemplary multimedia processors for performing such data formatting include the 

29 VCPEX chip by 8x8, Inc. of Santa Clara, California or digital signal processing chips by Texas 

30 Instruments and others. The audio/video codec 200 further comprises a field programmable gate 
array, electrically programmable read-only memory and random access memory for processing and 
packaging or formatting signals for compression and decompression. 

The sensor 104 may comprise a commercially available pulse oximeter sensor or other type 
of bio-sensor. A pulse-oximeter sensor allows the measurement of pulse rate and oxygen saturation 
of the blood. Data from the sensor 104 is transmitted to the general purpose node 150-1 50a, and 
transmitted to any remote media device connected to any of the networks 1 70-172. The sensor 104 
may comprise an "on body" wireless human performance and fatigue monitoring system that 
communicates with a belt-mounted transceiver/control module. The remote media device may 
comprise a processor 180-182 for display or logging of the real-time sensor signals. 

The headset 102a comprises a heads-up display 120 comprising a transparent color LCD 
device for video signals received and processed or formatted by the codec 200_for presentation to 
the user through the headset. The headset 102a may further comprise, or have attached thereto, an 
integrated microphone 122 for receiving voice commands from the user of the portable access unit 
100a or for communicating voice signals to a remote portable access unit 100 or remote media 
device through the general purpose node. The headset may further comprise a speaker 124 or 
earpiece unit for presenting audio signals to the user. The portable access unit 100a may further 
comprise a digital camera 1 06 that may either be attached on the user's person, or to the headset 1 02a 
for providing video signals to other portable access units 100- 100a or media devices which are 
routed through the general purpose node. 

With reference to Fig. 3, a flow diagram illustrating the method performed by the system of 
Fig. 1 is shown. 

A user puts on the data interface headset 102a of the portable access unit 100a, step 400. 
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The user may log into the local general purpose node 150 wherein the portable access unit 
associates with the general purpose node 150 such that the user is added to a connection list stored 
in a random access memory device residing in the general purpose node 150, step 401. 

Data is provided from the general purpose node 150 to the portable access unit through the 
communication devices 202, step 402. 

The user is presented with a selection list 190 of portable access units 100- 100a and media 
devices logged onto the system on the display 120, step 404. 

The user selects one of the entries from the selection list, step 406. 

A command corresponding to the selection is received by the codec and formatted for 
transmission by the communications device of the portable access unit by compressing and 
multiplexing the data. The formatted command is transmitted to the general purpose node 1 50 over 
the wireless connection, step 408. The general purpose node 150 establishes a network 170-174 
connection and routes the command to a network device for channeling data between the portable 
access unit 100 and the selected network device, step 410. 

The selected network device and the general purpose node exchange signals including the 
user's command, step 412. The network device may comprise the processor 180 or other network 
client 1 82 for running a software application, a camera 1 25 for providing remote viewing operations 
to the user on the display 120, the Internet phone 104 for providing voice communications with the 
remote user, or another portable access unit 1 00- 1 00a over a remote general purpose node 1 50a. By 
providing control commands to the microphone 122 or other input system, such as a keyboard or 
handheld mouse of the data interface headset, the user may issue commands to the general purpose 
node 1 50. The commands are formatted by the codec, transmitted to the general purpose node, and 
routed to the selected media or network device or other portable access unit 

The selected device executes the command. The result or data generated by the device is 
provided from the device to the general purpose node. The general purpose node routes the result 
to the corresponding personal access unit through the general purpose node and over the wireless 
connection. Upon receiving the result, the codec in the portable access unit formats the received data 
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by decompressing and demultiplexing the data. The formatted data can be video or audio data. The 
data is then presented to the user through a display, microphone, or other data interface component. 

It will thus be seen that changes may be made in carrying out the above system and method 
and in the construction set forth without departing from the spirit and scope of the invention, it is 
intended that any and all matter contained in the above description and shown in the accompanying 
drawings shall be interpreted as illustrative and not in a limiting sense. 
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ABSTRACT 

The system of the present invention is a highly integrated radio communication system 
with a multimedia co-processor which formats commands and data to permit true two-way 
multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager- 
sized portable access unit. The system is integrated in a network structure including one or more 
general purpose nodes which are coupled to network or media devices through a network for 
providing a wireless-to-wired interface. The network architecture allows video, audio and data 
(including biomedical data) streams to be connected directly to external users and devices 
through the general purpose nodes. The portable access units may also be mated to various non- 
personal devices such as cameras or environmental sensors for providing a method for setting up 
wireless sensor nets from which reported data may be accessed through the portable access unit. 
The reported data may alternatively be automatically logged at a remote computer for access and 
viewing through a portable access unit, including the user's own. 

GMV PAS434651.1-*-5/10/02 3:36 PM 
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WIRELESS AUGMENTED REALITY COMMUNICATION SYSTEM 
RELATED APPLICATIONS 
This application is based on provisional patent application serial number 60/1 15,993 filed 
January 15, 1999. 

GOVERNMENT LICENSE RIGHTS 

The U.S. Government has certain rights in this invention pursuant to NAS7- 1 407 awarded by NASA. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The invention is a wireless augmented reality system (WARS) that leverages communications 
and multimedia information processing microelectronics, along with displays, imaging sensors, 
biosensors, and voice recognition to provide hands-free, tetherless, real-time access and display of 
network resources, including video, audio and data. 

2. Description of the Prior Art and Related Information 

Online instruction manuals are becoming more prevalent in the industrial and everyday 
environment. These electronic technical manuals (ETM) may be interactive. Just as with printed 
manuals, ETMs may become very difficult to use and maintain in these environments where 
elements of an environment, such as dust, chemical or general harshness may be detrimental to the 
electronics and storage devices used to display and operate the ETM. Further, it is not always 
possible for a worker who requires access to an ETM to stop work to consult ETM. 

These problems are multiplied in extraterrestrial environments such as a space shuttle or a 
space station. During intra and extra vehicular activities, it may be virtually impossible to access a 
traditional keyboard and computer display to access an ETM. For example, during a satellite repair 
mission, it would not be practical for an astronaut in a bulky extravehicular space suit to type 
commands on a keyboard to view a display in the extreme environment of outer space where the sun 
glare may make viewing impossible. 

Hands- free portable computers have been implemented in an attempt to solve some of these 
problems. For example, U.S. Patent Nos. 5,305,244 and 5,844,824 describe systems in which a 
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head-up display and voice recognition is implemented in a portable computer for displaying ETM. 
However, these systems, being a single user-to-computer paradigm, do not allow multiple-user 
access to multiple computers, multimedia devices or nodes on a network for accessing arbitrarily- 
selected data channels. Further, these previously-described systems are self contained and their data 
storage needs to be updated periodically to be sure that the latest data is displayed. Further, these 
systems do not allow two-way communication over local and wide area networks to other mutli- 
media users and devices, and do not provide real-time biomedical information about the physical 
condition of the user. 

There is thus a need for a wireless, wearable communications system allowing two-way 
voice, video and data communication between local users and to remote users and devices over 
network nodes, along with tetherless real-time monitoring of the local user ! s physical condition. 
SUMMARY OF THE INVENTION 

The system solves the above problems with prior art systems with an adaptive wireless 
remote access network comprised of small individual portable access units which communicate with 
the general purpose node [ linked to a local cellula r g e neral pu r p o s e node ] through a wireless cellular 
connection . Media devices are coupled to the network and execute user commands. Interlinked 
general purpose nodes support communications across different habitat modules or internal-to- 
extravehicular communications, in the case of the space environment; terrestrial wired networks such 
as the Internet can serve as the interconnection of remotely scattered access nodes in an industrial, 
commercial or home environment application. 

The system may provide shuttle and international space station astronauts with tetherless, on- 
demand access to data channels from multimedia devices such as cameras or audio sensors 
associated with other persons or in a stand-alone configuration, and multimedia or data display from 
a networked computer terminal and to the equipment control capabilities which may be available 
through that computer. Transparent to such access, the system can maintain a data channel for 
monitoring an astronaut's health or environment via in-situ sensors. Though this system may be used 
for the shuttle and the international space station, the system has uses in many possible applications 
related to medical, industrial, and commercial areas. 
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The invention is a personal communications system designed especially for the space shuttle 
or station environment to provide cellular communications access throughout the vessel with video, 
audio, data and computer connect services. A small, wearable portable access unit (PAU) serves as 
a data interface and communicates over a wireless high-rate link to a centrally-located network 
access unit, called a general purpose node herein , for routing data and commands to/from the user . 
The system backbone provides 2-way video, 2-way audio, and a multi-purpose data channel between 
the PAU and general purpose node. One embodiment of the PAU used for personal communication 
has an attached headset with video display, audio feed and camera, which together may be used for 
audio or video teleconferencing. When used as a virtual terminal to a computer in the network, the 
user is able to view and manipulate imagery, text or video, using voice commands to control terminal 
operations. This can be accomplished by submitting a command through the general purpose node 
to a device which executes the command . 

Using the system, an astronaut may efficiently operate and monitor computer-controllable 
activities inside or outside the vehicle or station. Hands-free access to computer-based instruction 
texts, diagrams and checklists replaces juggling manuals and clipboards, and tetherless computer 
system access allows free motion throughout a cabin while monitoring and operating equipment. 
Along with voice commands, an integrated "touchpad" on the PAU may be used to submit user 
commands through the general purpose node and to any media devices, for remote computer control 
through a sensor data channel_[rthis] This return data channel may also be used [for] to transmit 
other control data [as] from a three-D mouse or data glove input device, through the general purpose 
routing node, to control a device coupled to the routing node through the network, allowing the real- 
time video display to b e us e d fo r r e mote, wi r eless remotely monitor and control [of] robotic cameras 
or manipulators. 

Concurrent with information provided to the astronaut, the system also allows external 
observation of the astronaut's situation; personal biological or other sensors can send back 
continuous telemetry through personal access unit and general purpose node. A miniature camera 
integrated into the headset provides real-time video of the wearer's field of view to remote observers. 
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In this way, for example, a principal investigator located on Earth may consult with a payload 
specialist on the operation or troubleshooting of their equipment. 

The system provides wireless high-rate data exchange. The radio link is adapted to operate 
within a high-interference, high-multipath environment of a space shuttle or space station module. 
Radio frequency (RF) links do not require visual line-of-sight to operate, but the metal walls and lack 
of RF absorbers, combined with moving human bodies, creates an enormous potential for destructive 
self-interference of the radio signals. The integrated radio and multimedia data processing 
technology provides for efficient and high-quality video and audio data compression for noisy indoor 
communications channels. The system supports multiple-user access for video, audio, and sensor 
data services in the wireless coverage area. Potential applications of the system are in any 
environment where heads-up, hands- free information retrieval or multimedia communications access 
improves efficiency including tetherless operations/monitor consoles, remote consultations in 
medical or maintenance procedures, and hazardous/confined space activities. There are also in-the- 
home entertainment/communications applications. 

Similar to the space extravehicular activities applications, bio-isolation suits have similar 
operation constraints to space suits. They are worn commonly where there are chemical or biological 
contaminates, and any extraneous materials brought into a chamber, such as clipboards or 
documents, also present a contamination risk. A unit suitably modified for use in a space suit could 
be used in this situation. This allows the user to use a computer (log data, use a check list, etc.), to 
communicate with colleagues, including providing first-hand video of work in progress, and to 
maintain constant monitoring of the health of the user. 

An extension of the medical applications areas would be in remote telemedicine. Many 
medical diagnostic and treatment tools are being made portable and rugged enough to be taken to 
remote sites. Some examples are an ultrasound unit that is the size of a backpack, an entire intensive 
care unit of equipment built into a stretcher, and a trauma pod built into a cruise missile. For many 
of these devices, CRT or LCD panels comprise a significant amount of the bulk and weight of the 
devices. The system of the present invention may provide a replacement for the CRT or LCD panel 
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as well as an interface to the control system of the device, while providing communications access 
through an interface to the remote site's existing communications equipment. 

Industrial applications include use by inspection or maintenance crews in remote or 
dangerous environments such as oil refineries, drilling figs, power plants, etc., where the personnel 
can move around with their hands and peripheral vision free to attend to their own safety and tasks. 
They would be in constant contact with the information they needed and any technical assist could 
be given by individuals looking at the return video images from the user. 

An example of a commercial application is for mission control and other operations 
personnel who presently must sit at a display console for hours at a time. These individuals could 
make use of the system of the present invention to increase their mobility and efficiency. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a diagrammatic illustration of the components of the system of the present invention; 
Fig. 2 is block diagram illustrating communications components used by the personal access unit 
and general purpose node of the system of Fig. 1; and 

Fig. 3 is a flowchart illustrating a method performed using the system of Fig. 1 . 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

With reference to Fig. 1, a diagram illustrating components of the system of the present 
invention is shown. The system may comprise small pager-like devices called portable access units 
100. The portable access units 100 [ ar c acc e ssorizablc f or] serve as different "multimedia" data 
interfaces for display, camera, audio and sensor operation. Another embodiment of the portable 
access unit 100a comprises a wearable headset and microphone assembly 102a. 

The portable access units 100- 100a interface directly through wireless link with a network 
through a general purpose node 150. The general purpose node 150 allows wireless-to-wire 
communication with a local network 1 70. The local area network 1 70 may be electrically connected 
to a wide area network or Internet 172 in order to connect to remote local area networks 174. 
Alternatively, the general purpose node 1 50 may be directly connected to the wide area network 1 72. 
The general purpose node 1 50 may thus act as a router for routing video, display, audio and control 
data packets between the portable access units 100 and other, or remote, portable access units 100 
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or remote media devices 125, 180, etc. connected to the networks 170-174. The connection with a 
network 170-174 may occur directly in electrical connection with one of the networks 170-174, or 
in wireless communication through a remote general purpose node 1 50a that is electrically connected 
to the network. The portable access units 1 00 may provide communication to and from remote media 
devices [ c o mp r ising ] such as computers 1 80-1 82 through the general purpose node which routes the 
communication to the corresponding media device . The computers 180-182 may be running 
specialized client software or certain commercial multimedia Internet software products such as 
video conferencing products that adhere to the industry standard H.323 for multimedia data transfer. 

Each portable access unit 100- 100a may dynamically associate with the closest general 
purpose node 150-150a when it is logged on to the network 170-174 or is connected thereto. Each 
general purpose node 1 50-1 50a records the associations and registers each portable access unit 100- 
100a on a list of connections associated with the particular general purpose node 1 50- 150a. The list 
of connections is stored in a random access memory device included in the general purpose node 
1 5 0- 1 5 0a. When a portable access unit 1 00 is logged off or disconnected from the network 1 70- 1 74, 
it is disassociated from the particular general purpose node 1 50- 1 50a that it was associated with, and 
thus, is removed from the list of connections. 

As shown on an example selection list screen 1 90 that may be presented on a display 1 02 or 
headset 1 02a on any of the portable access units 1 00- 100a, the user can set up a video, audio, or data 
link with any other portable access unit 100-100a or remote media device 125, 180, etc, that is 
logged onto a network 1 70- 1 74 through the general purpose node . The headset 1 02a may comprise 
a heads-up display ( 1 20 in Fig. 2) inside a headset embodying a transparent color LCD device. Using 
control keys or voice commands, a user of the portable access unit 100- 100a may select a local or 
remote portable access unit 100- 100a on a selection list 190 of other portable access units 100- 100a 
or media devices 125,180. The selection list 1 90 comprises a combination of the lists of connections 
stored on all of the general purpose nodes 1 50- 1 50a. Users may further access a nameserver located 
on the access node 150 for locating remote unfamiliar portable access units 100-100a or remote 
media devices. 
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By selecting entries from the selection list 1 90, users may communicate with portable access 
units 100- 100a or various media devices such as cameras 125, internet phones 104, one or more 
computers 180-182 located throughout the networks 170-174, through the general purpose node. 
The general purpose node routes the communication from the portable access unit, to a network or 
media device, or to another portable access unit . A user may further select from the list 190 user 
names representing users of other portable access units 100 that are logged in and associated with 
remote general purpose nodes 150a connected to the networks 170-174. 

With reference to Fig. 2, the components of the access node 150 and the wearable headset 
embodiment of the portable access unit 100a is shown. The portable access unit includes a display, 
an encoder and a decoder or "codec", and a two-way communications device or transceiver. 
[ Elements fo r b o th the ] The general purpose access node [ and portabl e acc e ss unit 100a include ] also 
includes a two-way communications device 202 . such as a transceiver. As a result, the portable 
access unit and general purpose node can communicate through a wireless connection through 
respective transceivers . Data processing functions are implemented in the form of an audio/video 
coder/decoder (codec) pair 200, one codec 200 comprising part of the portable access unit 100a A and 
the other codec 200 being part of another portable access node 100a or remote media device for 
which it is desired to exchange signals. At a portable access node, the codec 200 controls a digital 
data stream by formatting a command by the user into a form that can be transmitted by the 
communications device 202, [ which is fed t o the c o mmunicati o ns device 202, ] which is 
implemented as an RF modem transceiver pair with an equivalent communications device 202 
located in the general purpose access node. The general purpose node is configured to receive the 
formatted command from the transceiver of the portable access unit and transmit and route it using 
its own transceiver over the wireless connection. The codecs 200 serve as the interfaces to the 
external elements (including possibly the user display 1 02a and the sensor 1 04) on both sides of the 
communication continuum comprising the communications device 202 of the general purpose node 
150, an internal network interface protocol circuit 152, the external networks 170-174 and the 
electrical connection or general purpose access node connection to the desired remote portable access 
node or media device. The internal network interface protocol circuit 1 52 may comprise an Ethernet 
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chip, memory and a network protocol chip. With this architecture, the system addresses the issues 
of multiple-access and data channel quality, through the implementation of the communications 
device 202. Multiple implementations of the communication device 202 in the general purpose node 
150 allows for multiple simultaneous communication links with a plurality of portable access units 
100- 100a [for] through the general purpose node 150. 

With the base functionality of the communications device 202 and codec subsystem 200, the 
architecture provides flexibility in utilization of different external components such as different 
headset 102a configurations, sensor 104 packages, and network interface 152 capabilities. 

The communication device 202 is designed to operate in a high multipath space station or 
terrestrial indoor environment while being able to support multiple users at high, multimedia-type 
bandwidths. Thus the communications device's 202 physical (PHY) and media access (MAC) layers 
in combination support multiple access, dynamic network association, channel error rates of 
broadcast video quality (lxlOe-6), and data rates up to broadcast-quality video bandwidths (on the 
order of 768 kbps per user (one-way)). Modulation to achieve this performance will be differential 
phase-shift keying, of binary or higher order (quadrature or high-order quadrature amplitude 
modulation); the order chosen reflects the necessary user data volume to be supported within fixed, 
FCC-specified bandwidth allocations. Orthogonal frequency division multiplexing, code division 
multiple access, and frequency hopping/time division multiple access may be used for achieving 
multiple access. Spread spectrum, channel equalization, antenna diversity and retransmission 
techniques may also be used for improving the reliability of the communications link. Through a 
combination of these technologies, two-way multimedia channel throughputs can be achieved for 
each of multiple simultaneous users. A variety of RF frequencies may be used, but the determining 
factor in frequency band selection is the availability in the band of a relatively large amount of 
spectrum in the space station or FCC terrestrial allocations, allowing the transmission of compressed 
video. Ranges in the 2.5 to 5.7 band range are preferable due to the FCC bandwidth available, the 
compactness of RF elements required at these frequencies, and the potentially low amount of 
interference that will be sustained. The RF front end of both the portable access unit 100- 100a and 
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general purpose node 150- 150a may be interchangeable with different frequency front ends for 
system use in different 2 frequency bands. 

Low-rate, single user implementations of the communications system may be effected 4 
through adapted commercial wireless-LAN type products following the FCC 802. 1 1 standard 5 such 
as a frequency-hopping 2.4 GHz wireless LAN transceiver by Waveaccess, Incof 6 Wellesley, MA, 
or direct-sequence spread-spectrum 2.4 GHz wireless LAN chipset by Harris 7 Prism of Melbourne, 
FL. These radio implementations, as with commercial implementations of 8 the industry-proposed 
Bluetooth and HomeRF standards, will be limited in user access and 9 overall throughput, however, 
and therefore unsuitable to real-time video teleconferencing for 10 multiple users. The preferred 
embodiment for full capability is to implement the communications devices' physical and media 
access control layers in custom ASIC circuits 

allowing for support of all system capabilities within microelectronics architecture for small size 13 
and low power draw, providing pager-type form factor of wearable personal access units 10014 100a. 

The communications device 202 comprises a buffer memory and a radio frequency front 16 
end. Data modulation/demodulation circuits and error detection/correction protocol circuits are 17 
further included. Various combinations of these circuits may be obtained from Proxim of 18 
Sunnyvale, California, Harris of Melbourne, Florida and Stanford Telecom of Stanford, 19 
California. Alternatively, all of the various circuitry may be implemented with an application 20 
specific integrated circuit (ASIC), or a combination of an ASIC and discrete elements for size 2 1 and 
weight efficiency. 

Three classes of headsets 102a may be used: hi-resolution military systems which are CRT 
based and may be provided by Honeywell of Morristown, New Jersey, or Hughes Network Systems 
of San Diego, California; medium resolution industrial systems which are CRT or LED based 
scanners and may be provided by Intervision of Santa Clara, California; or low to medium resolution 
entertainment systems which are color viewfinder LCD based systems that may be 27 supplied by 
Virtual Vision of Redmond, WA (the V-CAP and E-GLASS), Sony Europe of 28 Hampshire, United 
Kingdom (GLASSTRON VISOR) or Olympus of San Jose, California. 29 Typical headset display 
120 specifications for the portable access unit 100a include the 30 following: 
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- RESOLUTION: Comparable at least to VGA (640x480) or better to 1 280 x 1 024 w/off-the- 
shelf display & I/O configuration 

- DISPLAY: >10 FL/day, Display Bright.Ratio:>2,Brightness range:200Mmax 

- FOV: 40-60 deg ,Gray scale: >12 

- EyeRelief: 20-26 mm TSP, 14/10mm(on/off-axis) exit pupil 

- Unif: 2:1 across 2/3 FOV, GLARE:< 2.5% image content,PixelContrast:25 

- FOCUS: Hands off, Obs:, % look- around ,Diopter range: 4-2, 

- Mag: 1+ p5%,Cont: >95%, motion sensor 10°cone, Inter. Eye. adj: 52-72mm 

- Image Enhan & IFF: Weaponsight, motion sensor and computer interface 

The audio/video codec 200 in a portable access unit 100- 100a or other client device is based 
around a single chip, standards-based codec that facilitates transmission of user commands over the 
network and receiving data or results in response thereto. 

When transmitting data, the codec formats data or commands for transmission bv the 
communications device over the wireless connection to the general purpose node. More specifically, 
the codec [that] accepts analog or digital audio and video (i.e. NTSC or VGA^compresses this 
input, and multiplexes the compressed data with an external data stream. The preferred industry 
standards are: ITU H.263 based video, ITU G.722 based audio, and ITU H.221 based multiplexing. 
The audio video codec 200 in the portable access unit 100- 100a can establish a link with a similar 
audio/video codec 200 associated with another portable access unit 100- 100a or a remote media 
device 104, 125, 180 or 182 through the general purpose node . 

When receiving data over the wireless connection through the general purpose node, the [The 
signals from the ] codec 200 in the portable access unit 100a formats the data routed through the 
general purpose node bv decompressing or demultiplexing the data and outputting [outputs! the 
received [ and decompressed re mot e signals ] data from the device for which the link was established 
to the user on the portable access unit . 

The interface between the codec 200 and communication device 202 as well as between the 
communication devices 202 of the general purpose node 150-1 50a and portable access unit 1 00- 1 00a 
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operate two-way with a high bandwidth suitable for transmitting video. Of this [ bandwith ] 
bandwidth , the audio portion utilizes up to 64 kbps and the data from the sensor 104 utilizes the 
required amount for the type of sensor 104, with the remainder allocated to compressed video. The 
quality of the video at these data rates in excess of 128 kbps is at least equivalent to video 
teleconferencing quality video. 

The audio/video codec 200 portion of the portable access unit 1 00- 1 00a may further comprise 
video input and output ports, audio input and output ports, data input and output ports ,and a the . The 
above-mentioned multimedia processor chip of the codec can format data retrieved through the 
general purpose node. The codec can be used for packaging signals for data compression and 
decompression for transmission. Exemplary multimedia processors for performing such data 
formatting include the 29 VCPEX chip by 8x8, Inc. of Santa Clara, California or digital signal 
processing chips by Texas 30 Instruments and others. The audio/video codec 200 further comprises 
a field [ pr o cess or] programmable gate array, electrically programmable read-only memory and 
random access memory for processing and packaging or formatting signals for compression and 
decompression. 

The sensor 104 may comprise a commercially available pulse oximeter sensor or other type 
of bio-sensor. A pulse-oximeter sensor allows the measurement of pulse rate and oxygen saturation 
of the blood. Data from the sensor 104 is transmitted to the general purpose node 150- 150a, and 
transmitted to any remote media device connected to any of the networks 170-172. The sensor 104 
may comprise an "on body" wireless human performance and fatigue monitoring system that 
communicates with a belt-mounted transceiver/control module. The remote media device may 
comprise a processor 180-182 for display or logging of the real-time sensor signals. 

The headset 102a comprises a heads-up display 120 comprising a transparent color LCD 
device for video signals received and processed or formatted by the codec 200 for presentation to 
the user through the headset . The headset 102a may further comprise, or have attached thereto, an 
integrated microphone 122 for receiving voice commands from the user of the portable access unit 
100a or for communicating voice signals to a remote portable access unit 100 or remote media 
device through the general purpose node . The headset may further comprise a speaker 124 or 
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earpiece unit for presenting audio signals to the user. The portable access unit 100a may further 
comprise a digital camera 1 06 that may either be attached on the user's person, or to the headset 1 02a 
for providing video signals to other portable access units 100- 100a or media devices which are 
routed through the general purpose node . 

With reference to Fig. 3, a flow diagram illustrating the method performed by the system of 
Fig. 1 is shown. 

A user puts on the data interface headset 102a of the: portable access unit 100a, step 400. 

The user may log into the local general purpose node 150 wherein the portable access unit 
associates with the general purpose node 150 such that the user is added to a connection list stored 
in a random access memory device residing in the general purpose node 150, step 401. 

Data is provided from the general purpose node 150 to the portable access unit through the 
communication devices 202, step 402. 

The user is presented with a selection list 190 of portable access units 100- 100a and media 
devices logged onto the system on the display 120, step 404. 

The user selects one of the entries from the selection list, step 406. 

A command corresponding to the selection is received by the codec and formatted for 
transmission by the communications device of the portable access unit bv compressing and 
multiplexing the data. The formatted command [ s e l e ction ! is transmitted to the general purpose 
node 150 over the wireless connection , step 408. The general purpose node 150 establishes a 
network 1 70- 1 74 connection and routes the command to a network device [ s e ts up a connecti o n o ve r 
th e networks 170-174 ] for channeling data between the portable access unit 100 and the selected 
network device, step 410. 

The selected network device and the general purpose node exchange signals including the 
user's command, step 412. The network device may comprise the processor 1 80 or other network 
client 1 82 for running a software application, a camera 1 25 for providing remote viewing operations 
to the user on the display 120, the Internet phone 104 for providing voice communications with the 
[z] remote user, or another portable access unit 100- 100a over a remote general purpose node 1 50a. 
By providing control commands to the microphone 122 or other input system, such as a keyboard 
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or handheld mouse of the data interface headset , the user may issue commands [ conduct o p e r ations 
by transmitting commands betw e en the portable access unit 100a and ] to the general purpose node 
150 . The commands are formatted by the codec, transmitted to the general purpose node, and routed 
to the selected media or network device or other portable access unit [ which routs the c o ntrol 
c o mmands to th e d e vice that th e use r selected, ste p 412 ]. 

The selected device executes the command. The result or data generated by the device is 
provided from the device to the general purpose node. The general purpose node routes the result 
to the corresponding personal access unit through the general purpose node and over the wireless 
connection. Upon receiving the result, the codec in the portable access unit formats the received data 
bv decompressing and demultiplexing the data. The formatted data can be video or audio data. The 
data is then presented to the user through a display, microphone, or other data interface component. 

It will thus be seen that changes may be made in carrying out the above system and method 
and in the construction set forth without departing from the spirit and scope of the invention, it is 
intended that any and all matter contained in the above description and shown in the accompanying 
drawings shall be interpreted as illustrative and not in a limiting sense. 
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ABSTRACT 



The system of the present invention is a highly integrated radio communication system with 
a multimedia co-processor which aHows formats commands and data to permit true two-way 
multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager-sized 
portable access unit. The system is integrated in a network structure including one or more general 
purpose nodes which are coupled to network or media devices through a network for providing a 
wireless-to-wired interface. The network architecture allows video, audio and data (including 
biomedical data) streams to be connected directly to external users and devices through the general 
purpose nodes . The portable access units may also be mated to various non-personal devices such 
as cameras or environmental sensors for providing a method for setting up wireless sensor nets from 
which reported data may be accessed through the portable access unit. The reported data may 
alternatively be automatically logged at a remote computer for access and viewing through a portable 
access unit, including the user's own. 
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